The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jan 30, 2026 7:29pm

Greptile Overview

Greptile Summary

This PR fixes a critical bug where multi-byte UTF-8 characters (Turkish characters, emojis, CJK characters) were being corrupted during SSE streaming when split across HTTP chunk boundaries.

Key Changes:

• Added { stream: true } option to TextDecoder.decode() in sse.ts:66, enabling the decoder to maintain state across chunks and properly handle incomplete UTF-8 byte sequences
• Implemented SSE message buffering using a buffer variable that accumulates partial SSE messages and splits on \n\n delimiters, keeping incomplete messages for the next chunk
• Added logic to flush remaining bytes when the stream completes by calling decoder.decode() without arguments
• Added comprehensive test suite with 25 tests covering various UTF-8 scenarios: Turkish characters, emojis, CJK characters split at 2/3/4-byte boundaries, and SSE message buffering edge cases
• Upgraded turborepo from 2.7.4 to 2.8.0
• Removed redundant comment in response-format.ts

The fix correctly handles both problems:

1. UTF-8 byte boundary splits: The { stream: true } option tells TextDecoder to buffer incomplete multi-byte sequences internally
2. SSE message boundary splits: The buffer keeps partial messages (e.g., "data: {"chu") until the rest arrives in the next chunk

Confidence Score: 5/5

• This PR is safe to merge - it fixes a real bug with a proper solution and comprehensive tests
• The implementation correctly uses TextDecoder's streaming mode and adds proper SSE message buffering. The fix is minimal, focused, and well-tested with 25 comprehensive tests covering edge cases. All existing 3621 tests pass.
• No files require special attention

Important Files Changed

Sequence Diagram

sequenceDiagram
    participant Client
    participant readSSEStream
    participant TextDecoder
    participant Buffer
    participant Callbacks

    Client->>readSSEStream: body.getReader()
    readSSEStream->>TextDecoder: new TextDecoder()
    
    loop For each chunk
        readSSEStream->>readSSEStream: reader.read()
        
        alt Chunk received
            readSSEStream->>TextDecoder: decode(value, { stream: true })
            Note over TextDecoder: Maintains state for<br/>incomplete UTF-8 sequences
            TextDecoder-->>Buffer: Decoded text
            Buffer->>Buffer: Split by '\n\n'
            Buffer->>Buffer: Keep incomplete message
            
            loop For each complete SSE message
                Buffer->>Buffer: Extract "data: " content
                Buffer->>Buffer: JSON.parse(lineData)
                
                alt Valid chunk
                    Buffer->>Callbacks: onChunk(data.chunk)
                    Buffer->>Callbacks: onAccumulated(accumulatedContent)
                end
            end
        else Done
            readSSEStream->>TextDecoder: decode()
            Note over TextDecoder: Flush any remaining bytes
            TextDecoder-->>Buffer: Final text
        end
    end
    
    readSSEStream-->>Client: accumulatedContent